1. Field of the Invention
The present invention relates to the field of multimedia electronic systems More particularly, the present invention relates to methods for encoding and decoding multimedia information and the transmission thereof including systems which display multimedia content that is decoded from encoded bitstreams originating from storage media and/or from an electronic network.
2. Related Art
Audio/visual (AV) material is increasingly stored, transmitted and rendered using digital data. Digital video representation of AV material facilitates its usage with computer controlled electronics and also facilitates high quality image and sound reproduction. Digital AV material is typically compressed (xe2x80x9cencodedxe2x80x9d) in order to reduce the computer resources required to store and transmit the digital data. The systems that transmit multimedia content encode and/or compress the content to use their transmission channel efficiently because the size of the multimedia content, especially video, is very large. Digital AV material can be encoded using a number of well known standards including, for example, the DV (Digital Video) standard, the MPEG (Motion Picture Expert Group) standard, the JPEG standard, the H.261 standard, the H.263 standard and the Motion JPEG standard to name a few. The encoding standards also specify the associated decoding processes as well. The multimedia contents are typically stored on the storage media and are transmitted as bitstreams.
The H.261 and H.263 standards of compression are for audio and visual data. These standards are described by the ITU-T (International Telecommunication Union). The H.261 standard is used for TV conference systems, while the H.263 standard is used for mobile communication. The H.261 and H.263 standards adopt the hybrid coding of Motion Compensation (MC) and Discrete Cosine Transform (DCT). The details of these specifications are described in ITU-T recommendations which are well known.
JPEG is the compression standard used for still images. It is standardized in ISO-IEC/JTC1/SC29/WG1 documents. The JPEG standard uses transform coding (DCT) as is also used in the JPEG standard. The WG1 is now standardizing JPEG2000, which is a more efficient encoding technology for still image processing. The JPEG2000 standard uses wavelet transformation. Motion JPEG is the defacto standard for video processing. In Motion JPEG, each frame of the video is encoded by JPEG. It is used in the Digital Video (DV) standard, which is used in the market today. Interframe correlation is not used for the compression.
MPEG is the compression standard for audio, video and graphics information and includes, for example, MPEG1, 2, 4 and 7. It is standardized in the ISO-IEC/JTC1/SC29/WG11 documents. MPEG1 is the standard for encoding audio and video data for storage on CD-ROM devices (compact disc read only memory). The MPEG1 specification is described in the IS-11393 standard. MPEG2 is the standard for encoding, decoding and transmitting audio/video data for storage media, e.g., DVD (digital video disc), etc., and also for digital broadcasts. MPEG2 supports interlaced video while MPEG1 does not. Therefore, MPEG2 is used for high quality video displaying on TV units. The MPEG2 specification is described in IS-13818. The MPEG4 standard is used for encoding, decoding and transmitting audio, video and computer graphics data. It supports content based bitstream manipulation and representation. The specification is described in IS14496. MPEG7 is the standard of the meta information of multimedia (MM) contents. The example of. the meta data is data is describes or is related to the MM contents, such as, identification and/or other descriptions of the author, producer information, directors, actors, etc. The MPEG7 standard is currently under standardization, and is in draft form but available. The draft specifications are described in the ISO-IEC-JTC1/SC29/WG11 documents.
In the past, existing multimedia systems have been closed applications. One example of an existing closed multimedia information retrieval system is the DVD system. The audio/visual contents are encoded by MPEG2 and stored on a DVD storage media (xe2x80x9cdiscxe2x80x9d). The DVD player reads the bitstream from the DVD disc and decodes its bitstream and then displays the decoded contents on the associated TV set. The DVD player is xe2x80x9cclosed, xe2x80x9d because the decoder circuit within the DVD unit only needs to be configured with one decoding technique because all contents of the DVD disc are always encoded using a single compression technique. For example, the DVD player is only required to decode the bitstreams that are encoded using DVD. A set-top-box (STB) is also only required to decode the bitstreams from a group of broadcasters that use a predefined encoding technique that is developed specifically to be decoded by the STB. For example, if an STB received information that it could not decode (e.g., it was encoded using a different compression technique), the STB might crash during the decoding process. Therefore, in closed applications, all bitstreams are encoded to guarantee to decode on a known client system with a known decoding format.
However, recently, many companies produce many different types of digital multimedia systems using different compression techniques. So far, digital AV multimedia systems have only been used for professional productions. The progress of LSI and signal processing technologies has recently reduced the cost of the digital systems and, as a result, AV multimedia systems are being used in consumer products. Examples of such digital consumer products include Video CD, DVD, DV (Digital Video), Digital Still Cameras, etc.
Digital computer technology and digital network technology has also progressed in the last 10 years. Computers are connected to each other by digital communication networks and can share the resources and information of the network. The Internet became very popular and is now an indispensable tool for many business and home users. Users can access various kinds of information and services through the Internet. In the case of businesses, the Intranet is very popular. With respect to an Intranet, all the computers in the office are connected to each other and their users can share information and resources that are in the same office. These network technologies not only allow access to information or resources, but also allow the distribution of computing power to several terminals in the network. The distributed system is very useful for terminals having limited computational power because it is possible to reduce the computational power required to execute some applications.
Network technologies have also been introduced to the field of consumer electronics. The standardization of a high-speed network, such as the serial communication standard, IEEE 1394, has been very successful with many companies developing products for this communication standard. In view of the expansive use of the Internet, the Intranet and in view of the growing use of consumer electronics being connected together via the IEEE 1394 communication standard, many digital systems can now receive information from various different digital systems. When the server receives a request for digital information, it sends the information in the form of a digital bitstream. This causes a problem, however, in that the encoding format used by the sender (server) may not match the decoding format used by the receiver (client). In this case, the client may not be able to decode the requested multimedia content sent by the sender. What is needed is an efficient system that allows networked digital systems to receive and decode encoded bitstream information that was sent from various different digital systems.
Accordingly, the present invention provides a method and system for allowing networked digital systems to receive and decode encoded bitstream information that was sent from various different digital systems. The present invention includes a format converter that is able to automatically convert (transcode) encoded bitstream information from one format to another so that the client (receiver) is able to display the received information. The format converter of the present invention also automatically allocates the computer resources that are necessary for converting to the appropriate format.
A multimedia information retrieval system and method are described including a method and system for automatic format conversion. The invention includes a data structure (xe2x80x9ccontents informationxe2x80x9d) that is associated with each requested multimedia bitstream. The data structure identifies the encoding format, e.g., compression technique, used in the encoded multimedia bitstream which can be originated by a contents server. An automatic format conversion process then queries information from the client system (requester) and also receives the data structure identifying the encoding format. The client information identifies the decoding format compatible with the client.
The automatic format conversion then determines the transcoding process required for converting the bitstream from its encoded format to the format recognized by the client system. The format conversion process of the present invention also determines whether or not decoding is required before transcoding is performed thereby saving processing time and computer resources in those cases where decoding is not required. Moreover, the format conversion process also automatically determines the computer memory size required (frame memory) to perform the transcoding process thereby efficiently using computer memory resources. The format converter can be implemented in software as an application and can also be integrated within a data access server. The data access server can be integrated within the client system or within the contents server. The format converter of the invention is particularly useful for electronic devices coupled in a communication network where the encoding format of the sender may not be compatible with the decoding format of the receiver, thereby requiring transcoding between the formats.
One embodiment of the present invention includes a method for processing multimedia information comprising the steps of: a) responsive to a request signal from a client, accessing a portion of encoded multimedia information from a database storing encoded multimedia information; b) accessing contents information describing an encoding format of the portion of encoded multimedia information and accessing client information describing a decoding format of the client; and c) automatically transcoding the portion of encoded multimedia information to generate transcoded multimedia information based on the contents and client information, wherein the transcoded multimedia information is compatible with the decoding format of the client. Embodiments include the above and wherein step c) comprises the steps of: c1) based on the contents and client information, automatically determining whether or not decoding is required of the portion of encoded multimedia information; c2) provided decoding is required, automatically determining memory resources required to decode the portion of encoded multimedia information; c3) provided decoding is required, decoding the portion of encoded multimedia information to produce decoded multimedia information; and c4) based on the contents and client information, transcoding the decoded multimedia information to provide the transcoded multimedia information.